A STRATIFICATION ON THE MODULI SPACES OF 
SYMPLECTIC AND ORTHOGONAL BUNDLES OVER A CURVE 



INSONG CHOE AND GEORGE H. HITCHING 

| Abstract. A symplectic or orthogonal bundle V of rank 2n over a curve has an 

invariant t(V) which measures the maximal degree of its isotropic sub-bundles of 
£NJ ' rank n. This invariant t defines stratifications on moduli spaces of symplectic and 

orthogonal bundles. We study this stratification by relating it to another one given 

Oh' 

' by secant varieties in certain extension spaces. 

' We give a sharp upper bound on t(V), which generalizes the classical Nagata 

bound for ruled surfaces and the Hirschowitz bound for vector bundles, and study 
■ the structure of the stratifications on the moduli spaces. In particular, we compute 

the dimension of each stratum. We give a geometric interpretation of the number 
of maximal Lagrangian subbundles of a general symplectic bundle, when this is 
finite. We also observe some interesting features of orthogonal bundles which do 
not arise for symplectic bundles, essentially due to the richer topological structure 
of the moduli space in the orthogonal case. 
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1. Introduction 



> 

m 
oo 
q 

; Let X be a smooth algebraic curve over C. A vector bundle V of rank two over 

X determines a ruled surface PV. Such surfaces have been studied since the 19th 



century. A line subbundle L of V gives a section o L of PV, called a directrix curve. 
The self-intersection number of ctl is given by 

o L -a L = deg(V/L) — degL = degV — 2degL. 

The Segre invariant of PV is defined as the minimal value of ■ over all L C V. 
Via the above formula, this invariant also provides a measure of the difference of the 
slopes of V and L. 

The Segre invariant yields a natural stratification on the moduli space of vector 
bundles of rank two over X, which was studied by Lange and Narasimhan [I3J. A 
generalization of this stratification to the moduli of vector bundles over X of arbitrary 
rank was considered by Lange [TT] and the details were settled in Brambila-Paz-Lange 
[2], and Russo-Teixidor i Bigas [19] . The aim of the present article is to establish 
parallel results for symplectic and orthogonal bundles over X. 
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Let us define a few notions and fix notations. A vector bundle V over X of rank 2n 
is called symplectic (resp., orthogonal) if there is a nondegenerate alternating (resp., 
symmetric) bilinear form u: V <8> V — > Ox- A subbundle E of V is called isotropic if 
w\e®e — 0. By linear algebra, an isotropic subbundle of V has rank < n. When V is 
symplectic, a rank n isotropic subbundle of V is often called a Lagrangian subbundle. 
We say that a symplectic or orthogonal bundle V is stable (resp., semistable) if for 
every isotropic subbundle E of V, we have 

p<y) = ^r > w = (resp " MK) - 

Note that this is a priori weaker than the stability condition for V as a vector bundle; 
compare with Ramanathan [18J . However, Ramanan [T7] proved that semistability as 
an orthogonal bundle is equivalent to semistability of the underlying vector bundle, 
and moreover that a general stable orthogonal bundle is a stable vector bundle. The 
same argument (worked through in [5]) shows that the analogous statement is true 
for symplectic bundles. 

We denote by SUx(2n, Ox) the moduli space of semistable vector bundles of rank 
2n and trivial determinant, and write AiSx{2n) (resp., AiO x (2n)) for the sublocus 
in SUx{2n,Ox) of bundles admitting a symplectic (resp., orthogonal) structure. 

In the symplectic case it has been proven by Serman |20j that the forgetful map 

Mx(Sp 2n C)^SU x (2n,O x ) 

associated to the extension of the structure group Sp 2n C C SL 2n C, is an embedding, 
where A^x(Sp 2n C) is the moduli space of semistable principal Sp 2n C-bundles over 
X. So AiSx(2n) coincides with the embedded image of A^x(Sp 2n C). 
The orthogonal case is more delicate. By |20j, the forgetful map 

M x (S0 2n C) -> SU x (2n, O x ) 

is generically two-to-one, amounting to forgetting the data of an orientation on a 
principal S02 n C-bundle. On the other hand, the map 

M x (0 2n C) ^ M x {GUnC) 

is an embedding. The moduli space A / ix(C>2n ( C) of semistable principal 2n C-bundles 
over X has several components, which are indexed by the first and second Stiefel- 
Whitney classes (wi,w 2 ) G H X (X, Z 2 ) x H 2 (X, Z 2 ). The class Wi corresponds to the 
determinant, and there are two components of M.x{0 2n 'C) with w\ trivial. We write 
AiOx(2n) ± for the embedded images of these components in SUx{2n, Ox)- 
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Let V be a symplectic or orthogonal bundle of rank 2n. Generalizing the Segre 
invariant for bundles of rank two, we define 

t(V) := — 2max{deg.E : E a rank n isotropic subbundle of V}. 

In particular, if V is stable (resp., semistable), then t(V) > (resp., t(V) > 0). (Note 
that when V is a symplectic bundle of rank > 2, this differs from the invariant SL ag 
defined in [3] by a constant: t(V) = ^p[SL ag (y).) For a symplectic or orthogonal 
bundle V, we denote by M(V) the space of rank n isotropic subbundles E C V such 
that t(V) = —2 deg E. It can be viewed as a closed subscheme of a Quot scheme, so 
it has a natural structure of projective variety. 

The invariant t(V) induces stratifications on moduli spaces of semistable symplectic 
and orthogonal bundles over X. For each positive even integer t, we define 

MS x (2n;t) := {V G MS x (2n) : t(V) = t} 

and 

MO x (2n;t) := {V G MO x (2n) : t(V) = t}. 

By semi-continuity of the invariant t(V), these subloci are constructible sets. For the 
symplectic case, we show: 

Theorem 1.1. Let X be a smooth algebraic curve of genus g > 2. 

(1) For any symplectic bundle V of rank 2n, we have t(V) < n(g — 1) + 1. This 
is sharp in the sense that for a general V G M.Sx{2n), 

n(g-l)<t(V)<n(g-l) + l. 

(2) For each even integer t with 2 < t < n(g — 1), the stratum A4Sx(2n;t) is 
nonempty and irreducible, and 

dimMS x (2n;t) = - (n(3n + l)(g - 1) + (n + l)t) . 

Furthermore, if t < n(g — 1), then A4Sx{2n;t) is contained in the closure of 
MS x (2n;t + 2) mMS x (2n). 

(3) For each positive even integer t <n(g—l), the space M(V) is a single point for 
a general V G AiSx(2n; t). On the other hand, for a general V G M.Sx{2n), 



dimMfV) 



when n(g — 1) is even, 
when n(g — 1) is odd. 
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Let us give some historical remarks. For vector bundles, the Segre stratifications 
are well understood; see Lange-Narasimhan [13], Hirschowitz [4], Brambila-Paz- 
Lange [2] and Russo-Teixidor i Bigas [19]. Holla and Narasimhan [8] defined a 
generalized Segre invariant for principal G-bundles for an arbitrary reductive group 
G, and obtained a bound on the invariant in general. This bound was sharpened 
for symplectic bundles (G = Sp 2n C) by the present authors [3], where Theorem 11.11 
(1) was proven using the Terracini lemma in projective geometry. Also Theorem ll.il 
was proven for symplectic bundles of rank four in [3J. The special case when the 
curve has genus two has earlier been studied in detail by the second named author 
[7]. In this paper, we provide another simpler proof of (1), and prove (2) and (3) for 
arbitrary rank n and genus g. 

In the orthogonal case, the moduli space M.0x(2n) has two connected compo- 
nents, as was discussed before. We denote by M.0x(2n) + (resp. AiOx(2n)~) the 
component consisting of bundles of trivial (resp., nontrivial) second Stiefel-Whitney 
class. We first prove the following. 

Theorem 1.2. (1) Let E\ and E 2 be isotropic rank n subbundles of an orthogo- 
nal bundle of rank 2n. Then deg-Ei and deg-E^ have the same parity. 

(2) A semistable orthogonal bundle V belongs to M.0 x{2n) + (resp., AiO x{2n)~ ) 
if and only if its isotropic rank n subbundles have even degree (resp., odd 
degree). 

We then show the following on the stratification on each component. 

Theorem 1.3. Let X be a smooth algebraic curve of genus g > 2. 

(1) For any orthogonal bundle V of rank 2n, we have t(V) < n(g — 1) + 3. This 
is sharp in the sense that two even numbers t with n(g — 1) < t < n(g — 1) +3 
correspond to the values oftiV) for generalV in AiOx{2n) + and A40 x{2n)~ . 

(2) For each even integer t with 2 < t < n(g — 1), the stratum M.0x{2n\t) is 
nonempty and irreducible, and 

dim MO x (2n; t) = - (n(3n - l)(g - 1) + (n - l)t) . 

Furthermore, if t < n(g — 1) then AiOx{2n;t) is contained in the closure of 
AiOx{2n\ t + 4) in the relevant component A / 10x(2n) ± . 

(3) For each positive even integer t < n(g — 1), the space M(V) is a single point 
for a general V G AiSx(2n; t). 
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We also compute the dimension of M(V) for general bundles in each component 
A / 10x(2n) ± . It turns out to depend on the class of n(g — 1) modulo 4; the pre- 
cise statement is set out in §5.3. A more detailed description of the top strata of 
AiOx{2n) is given in §5.4. 

To prove these statements, we consider families of symplectic and orthogonal ex- 
tensions. The main strategy is to relate the invariant t(V) to the geometry of certain 
higher secant varieties in the projectivized extension spaces. This idea goes back 
to the work of Lange and Narasimhan [T3] , where Theorem 11.11 is proven for rank 
two bundles using higher secant varieties of the curve X embedded in the extension 
spaces. 

In §2, we generalize the geometric framework in [13] to our situation in Criterion 
12.81 The embedded curve X in the extension space is replaced by the quadric bun- 
dle FE and the Grassmannian bundle Gr(2, E) for symplectic and orthogonal cases 
respectively. To work with bundles of rank In > 4, one has also to understand the 
situation when two rank n isotropic subbundles intersect non-transversely. It turns 
out that this case can also be understood geometrically (Criterion 12.111) . This is a 
key advance upon the methods in [3J, which enables us to argue for symplectic and 
orthogonal bundles of arbitrary rank. 

In §3, we construct "universal extension spaces" parameterizing all the extensions 
of fixed type, and show that the rational classifying maps to JASx{2n) or AiOx(2n) 
are defined on dense subsets. In §4 and §5, we prove the main results for symplectic 
and orthogonal bundles respectively, using dimension counts based on the geometric 
information from the preceding sections. In §5, we also observe some interesting 
properties of certain families of orthogonal bundles, stemming from the richer topo- 
logical structure of the moduli space. 
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2011-A423-0004). Also he would like to thank H0gskolen i Vestfold, Bakkenteigen 
for the invitation in June 2011. The second author gratefully acknowledges the 
hospitality and generous financial support of Konkuk University, Seoul. Both authors 
thank Christian Pauly and Olivier Serman for helpful discussions. 
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2. Symplectic and orthogonal extensions and lifting criteria 

In this section we discuss symplectic and orthogonal extensions. Most of the results 
in this section were obtained for symplectic extensions in [31 §2]. Those results will be 
restated here for the reader's convenience, and the modified versions for orthogonal 
extensions will be proven in detail. 

Let V — » X be a vector bundle of rank 2n equipped with a nondegenerate bilinear 
form u: V <g> V — > Ox, and let E C V be a subbundle. Then there is an exact 
sequence 

-»• E L -»■ V -»> E* -»> 0, 

where -E -1 is the orthogonal complement of E. If E is isotropic of rank n, then 
E = E ± and V defines a class e ff^X, Hom(£'*, ^ ff^X, £ ® £). 

Criterion 2.1. Suppose E is simple. An extension — > E — > V — > E* — > is 

induced by a symplectic (resp., orthogonal) structure on V with respect to which E is 
isotropic if and only if the extension class S(V) belongs to the subspace 

H X {X, Sym 2 £) {resp., H\X,A 2 E)) 

ofH 1 {X,E®E). 

Proof. This is due to S. Ramanan. A detailed proof for the symplectic case is given 
in [6l §2], and the proof for the orthogonal case is practically identical. □ 

2.1. Cohomological criterion for lifting. Here we recall the notion of a bundle- 
valued principal part (see Kempf [TU] for corresponding results on line bundles). A 
locally free sheaf W on X has the flasque resolution 

->■ W ->■ Rat(W) ->■ Pnn(W) -> 0, 

where Rat (V^) is the sheaf of rational sections of W and Prin (W) the sheaf of W- 
valued principal parts. We denote their groups of global sections by Rat(PF) and 
PrinfW) respectively. Taking global sections, we obtain 

(2.1) ->■ H°(X, W) -> Rat(W) Prin(W) ->■ if^X, W) ->■ 0. 

For a principal part p G Prin(W / ), we write its class in H 1 (X, W) as \p\. 

Now consider an extension 0— > E — > V — > E* — > 0, and an elementary transfor- 
mation F of E* defined by the sequence 
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for some torsion sheaf r. We say that F lifts to V if there is a sheaf injection F — > V 
such that the composition F — > V — > E* coincides with //. The following statements 
are proven in (6J §3]: 

Lemma 2.2. Suppose h°(X, Hom(E*, E)) = 0, and consider an extension — > E — > 
V ->• E* with class 5(V) E H l (X, E®E). 

(1) There is a bisection between principal parts p E Prm(E®E) such that S(V) = 
[p], and elementary transformations of E* lifting to subbundles of V . The 
bisection is given by 

p <r> Ker (p: E* -> Pim (E)) . 

(2) Suppose that S(V) = \p) E ^(X, Sym 2 E), corresponding to a symplectic 
extension. The subbundle corresponding to Ker(p) is isotropic in V if and 
only if p is a symmetric principal part; that is, l p = p. 

(3) Suppose that S(V) = [p] E H 1 (X,A 2 E), corresponding to an orthogonal ex- 
tension. The subbundle corresponding to Ker(p) is isotropic in V if and only 
if p is an antisymmetric principal part; that is, l p = —p. □ 

2.2. Subvarieties of the extension spaces. Given any vector bundle W, consider 
the projectivization tt: FW — > X. Then we have a natural rational map 



defined as follows (a slightly different description was given in [3l §2.3]): 

Consider the evaluation map X x H°(X,K X ® W*) — > K x <8> W*. Via Serre 
duality, the dual of this map is identified with 



Projectivizing this map and then composing with the projection X x FH l (X, W) — > 
FH 1 (X, W), we get a map 



On a fibre W\ x , this map is identified with the projectivized coboundary map in the 
sequence 



FW — » FH\X, W) 



W®T x ->Xx H\X, W). 



(p: FW — > FH\X, W). 



(2.2) 

->■ H°(X, W) -> H°(X, W{x)) -> 



W{x) 
W 



X 



->■ H\X, W) ->■ H\X, W{x)) ->■ 0. 
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Thus for w G FW, the image <f)(w) may be realized as the cohomology class of a 
W- valued principal part supported at x with a simple pole along w. As discussed 
in [3j §2.3], the rational map is induced by the complete linear system of the line 
bundle n*K x ® ¥W {1) over FW. 

For W = Sym 2 E and W = A 2 E respectively, we have the rational maps 

P(Sym 2 £) — > Pi/^X, Sym 2 £) and F(A 2 E) — ► Ptf^X, A 2 E). 

Note that both of these are restrictions of the map F(E <g) E) — > Pif^X, E ® E). 
In the symplectic case, we consider the chain of maps 

FE m> P(Sym 2 £) — » Pi/^X, Sym 2 £), 

where the first inclusion is given by the Segre embedding [v] >->• [u <8> u]. In the 
orthogonal case, we consider the chain of maps 

Gr(2,£) ■=— )■ P(A 2 £) --4 Ptf^X, A 2 £), 

where the Grassmannian bundle Gr(2, is embedded in P(A 2 £') via the Pliicker 
embedding. These rational maps are denoted as 

(2.3) (j) s : FE — > Ptf^X, Sym 2 £) and a : Gr(2, £) — > Ptf^X, A 2 £). 

Lemma 2.3. Let W = E <g) £ /or a stable bundle E over a curve of genus g > 2. 

(1) The rational map 0: P(W / ) ---> Pif 1 (X, W) frase point free if fi(E) < — |, 
and an embedding if fi(E) < — 1. In particular, if fi(E) < — 1, the rational 
maps 4> s and a in A2.3\) are embeddings. 

(2) // .E zs general of negative degree, is injective on a general fiber ofF{W). 
If we further assume that g > 3 then separates two general fibers ofF(W). 

Proof. (1) (A similar version was stated in [31 Lemma 2.4], unfortunately with a 
flawed proof.) One can check that is base point free (resp., an embedding) if 
h°(X, W(D)) = for all effective divisors D of degree one (resp., of degree two) on 
X. Since W = E®E is semistable, these vanishing results follow from the assumption 
that fj,(E) < —\ (resp., fi(E) < -1). 

(2) Let L and M be line bundles with degL = deg-E and degM = 0, and put 
E = L® M®*"" 1 ', where n = rk(E). One can check that for a general choice of L 
and M, the bundle Eq £g> Eq(x) has no sections. Deforming Eq to a general stable 
bundle E, we see that h°(X,W(x)) = for a general x G X. A similar argument 
shows that if g > 3, then h°(X, W(x + ?/)) = for general x, y G X. □ 



SYMPLECTIC AND ORTHOGONAL BUNDLES 



9 



2.3. Irreducibility of the space of principal parts. In this subsection, we prove 
some technical facts on principal parts. Let p be a principal part with values in 
E® E. 

Definition 2.4. The degree deg(p) of p is defined as the length of the torsion sheaf 

Im(p: E* -» PrinQE)) . 

The support of p is defined as the support of Im(p). A symmetric (resp., antisym- 
metric) principal part p of degree k (resp., 2k) is called general if it is supported at 
k distinct points. 

Obviously, the general symmetric (resp., antisymmetric) principal parts of degree 
k (resp., 2k) are parameterized by a quasi-projective irreducible variety. We want to 
confirm that an arbitrary symmetric or antisymmetric principal part can be obtained 
as a limit of a continuous family of general ones. 

Lemma 2.5. ([3] Lemma 2.6]) Let p be a Sym 2 E-valued principal part of degree k, 
supported at a single point x. Then there exists a local frame e\, . . . , e n for E in a 
neighborhood of x, in terms of which p is expressed as 

n 

E ej ® Cj 

i=i 

where z is a uniformizer at x, and k\ > k 2 > ■ ■ ■ > k n > 0, and k = Y^i=i '— ' 

For antisymmetric principal parts, we have the following analogue: 

Lemma 2.6. Let p be a A 2 E-valued principal part, supported at a single point x. 
Then there exists a local frame e±, . . . , e n for E in a neighborhood of x, in terms of 
which p is expressed as 

s 

E&2i-1 A en 

i=l 

where z is a uniformizer at x, and k\ > k 2 > • • ■ > k s > 0, and deg(p) = 2 (X)i=i K)- 
In particular, any antisymmetric principal part has even degree. 

Proof. This argument is adapted from the proof of [31 Lemma 2.6]. Locally, p can be 
expressed as 

P = ~T A 
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for some nxn antisymmetric matrix A with entries in the ring R = C[z]/ (z kl ), since 
we are concerned with the principal parts at x only. In this context, it suffices to 
show that there exists a matrix P £ M n (R), such that det P is a unit in R and 

'PAP = diag(^ 1 J, z d2 J, . . . , z da J), 

where = d\ < d 2 < ■ ■ ■ < d s and J = ^ ^ \. This can be shown by mimicking 

the standard process to get the normal form of an antisymmetric matrix over C. The 
only difference lies in that R has non- units contained in the principal ideal (z), and 
this can be taken care of by allowing the terms z 1 , . . . , z ds on the diagonal. 

Alternatively, one may work instead over the formal power series ring Cffz]], which 
is a PID, and truncate the terms in the ideal (z hl ) at the final step. For the process 
over a PID, see Adkins-Weintraub [TJ Ch. 6, Corollary (2.36)]. 

Hence we have a frame for E on this neighborhood in terms of which p appears as 



(2-4) E 



e 2i -i A e 2i 



i=l 

where ki = k\ — di. Since the terms in the sum (12.41) impose independent conditions 
on sections of E, we have 

'e 2 i_iAe 2 i 



deg(p) = £ deg ( 62i X zh 62t ) =2$>, 

i=l i=l 

as required. □ 

Corollary 2.7. For a fixed vector bundle E and for each k > 0, the spaces of 
symmetric principal parts of degree k and antisymmetric principal parts of degree 2k 
are irreducible. 

Proof. We consider the antisymmetric case; the symmetric case can be proven sim- 
ilarly. It suffices to show the irreducibility of the space of antisymmetric principal 



parts supported at a single point x. By Lemma I2.6[ we may choose a trivialization 

of E near x with respect to which p is expressed as Yli=iPi> where 

e 2 i-i A e 2i 
Pi = k 

and 2 ^2t=i ^* = ^- For each i, choose distinct X\, . . . ,X l k . £ C, and define a family of 
principal parts 

e 2 ;_i A e 2i 

Pi{t) 



(z-\\t)---(z-\U) 
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where t is a complex parameter. We then put pit) = Y^i=iPi(^)- By construction, 
p(0) = p, while for small t ^ we can rewrite p(t) as a sum of k antisymmetric 
principal parts of degree 2, supported at k distinct points. Hence we have shown 
that any antisymmetric principal part of degree 2k is a limit of a continuous family 
of general antisymmetric principal parts of degree Ik. But we know that the gen- 
eral antisymmetric principal parts are parameterized by an irreducible variety. This 
proves the claim. □ 

2.4. Geometric criteria for lifting. In this subsection, we find a geometric inter- 
pretation of the cohomological criterion on isotropic liftings in Lemma 12. 2[ 

Throughout this subsection, E is a general stable bundle of negative degree. In 
particular, E is simple and h°(X, Hom(E*, E)) = 0. Consider the rational maps 

<f) s : P£7 — » FH\X, Sym 2 £) and a : Gr(2, E) — » FH\X, A 2 E). 

As discussed in Lemma [2T3l these maps are defined on dense subsets, and furthermore 
are embeddings if fi(E) < — 1. In general, abusing notation, we denote by WE and 
Gr(2,£) the closures of the images <p s (FE) in ¥H 1 (X, Sym 2 £) and a (Gr(2,£)) in 
¥H X {X, A 2 E) respectively. 

Next, for a quasi-projective variety Z C P^, we write Sec fc Z for the fc-th secant 
variety of Z, which is the closure of the union of linear subspaces spanned by k 
general points of Z. In particular, Sec 1 ^ = Z. 

Criterion 2.8. Consider an extension given by 

5(V) : 0^E-+V^E*^0. 

(1) When 8{V) e FH\X,Sjm 2 E), there is an elementary transformation F of 
E* with deg(E*/F) < k lifting to a Lagrangian subbundle ofV if and only if 
8{V) e Sec fc P£. 

(2) When 5{V) G FH 1 (X, A 2 E) , there is an elementary transformation F of E* 
with deg(E*/F) < 2k lifting to a rank n isotropic subbundle ofV if and only 
if 5(V) G Sec fe Gr(2, E). In this case, degE and degF have the same parity. 

Proof. In the symplectic case, this is the content of [31 Lemma 2.10 (2)]. The same 
idea works for antisymmetric case as follows: 
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By Lemma [2.21 (3), an elementary transformation F C E* with deg(E*/F) < 2k 
lifts to an isotropic subbundle of V if and only if the extension class S(V) is of the 
form \p\ where p is an antisymmetric principal part of degree 21 < k. 

If p is general, it is of the form 

(2.5) 

1=1 

where Z\, Z2, ■ ■ ■ , z\ are uniformizers at I distinct points x%, X2, ■ ■ ■ , %i G X. According 
to the description of the map <fr a in §2.2[ the class \p] G FH 1 (X, A 2 E) lies on the 
secant plane spanned by I distinct points of Gr(2,£ l ). Hence [p] G Sec'Gr(2, E). 
Conversely, a general point of Sec'Gr(2,i?) corresponds to a class 8(V) = [p] where 
p is of the form (12.51) . 

From this correspondence on the general points, we get the desired statement by 
Corollary 12.71 In this case, F = Ker(p: E* — > Prin (-E)) with degp = 21, so degF 
has the same parity as deg E. □ 

Corollary 2.9. Consider an extension given by 

5{V) : 0^E^V^E*^0. 

(1) Assume 6(V) G H\X, Sym 2 E). If6(V) < Sec k FE, thent(V) < 2(k + degE). 

(2) Assume 6(V) G H 1 {X,A 2 E). If 6(V) < Sec fe Gr(2, E), then t(V) < 2(2k + 
degE). 

Proof. By definition of the invariant, t(V) < — 2degF for any rank n isotropic sub- 
bundle F C V. The above bounds follow as a direct consequence of Criterion [2781 D 

The converse of the above Corollary is not true in general. For instance, assume 
6{V) G H l (X,$ym 2 E). The bound t{V) < 2{k + degE) tells us that there is a 
Lagrangian subbundle F of degree > — deg E — k, but this does not imply that F 
lifts from an elementary transformation of E*. So in general, we are led to consider 
a diagram of the form 

(2.6) E > V » E* 



H F ^ G ^ 

where H C E is a subbundle of degree —h < and rank r > 0, and G is a locally 
free subsheaf of E* of rank n — r. When r = so that E\ x and F\ x meet transversely 
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for general x G X, this reduces to the situation of Criterion 12.81 of the lifting of 
elementary transformations. The remaining part of this subsection will be devoted 
to finding a criterion for the existence of such a diagram with r > 0. 

For H C E, let q: E — > E/H be the quotient map. Since H is isotropic, H 1 - fits 
into the diagram 

»- E ^ V - E* » 

*q 

E H 1 (E/H)* 0. 

For G iJ 1 (X,Hom(E*, J E;)), we have 

= V(W) G H\X i Bam.{{E/B)\E)). 

Furthermore, if -1 inherits a (degenerate) bilinear form from V. Since (if -1 )- 1 - = H, 
the quotient H L /H is equipped with a nondegenerate bilinear form coming from 
V. In fact, H 1 - /H is a symplectic (resp., orthogonal) extension in the upper exact 
sequence of 

E/H * H L /H > (£/#)* 

o >■ e ^ (£/#)* — ^ o 

corresponding to the class 

8(H X /H) = W)) 
in H l (X,Sym 2 (E/H)) (resp., H 1 (X, A 2 (E/H)) ). 
Lemma 2.10. 27ie induced maps 

q*Cqy : # X pf, Sym 2 £) -> H 1 (X, Sym 2 (E/H)) 

and 

q^q)* : H\X, A 2 E) H 1 (X, A 2 (E/H)) 

are surjective. 

Proof. These maps are induced from 

Cq)* :E®E^ (E/H) ® E and q* : (E/H) ®E^ (E/H) ® (E/H) 
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respectively. By local computation, it can be seen that the images of q*(^q)* in 
(E/H) ® (E/H) are precisely Sym 2 (E/H) and A 2 (E/H) respectively. Hence on the 
first cohomology level, the induced maps are surjective. □ 

We obtain the following geometric criterion on liftings: 

Criterion 2.11. Let V be an extension of E* by E with class S(V). Fix a subbundle 
^ H C E and write deg(E) = -e, deg(F) = -h. Let f > 2h - e. 

(1) Assume 5(V) E H 1 (X,Sym 2 E). Then V admits a Lagrangian subbundle F 
of degree > — f inducing a diagram of the form A2.6\) if and only if 

q*?q)*(6{V)) E Sec {e+f - 2h) F(E/H). 

(2) Suppose S(V) E H 1 (X,A 2 E) and e = / mod 2. Then V admits a rank n 
isotropic subbundle F of degree > —f inducing a diagram of the form \2. 6]) 
if and only if 

q*Cq)*(8(V)) e Sec l ^ e+f - 2h ^Gi (2, e/h). 

Proof. Since the arguments for (1) and (2) are parallel, we prove (2) only. The 
orthogonal bundle V admits an isotropic subbundle F inducing the diagram (12.61) if 
and only if H 1 - admits a subbundle F of degree > — / which is isotropic with respect 
to the antisymmetric form inherited from V, yielding the diagram 

E ^ H L (E/H)* 

H F G 

Since G = F/H has the same rank as (E/H)*, the map G — > (E/H)* is an elementary 
transformation whose quotient is a torsion sheaf of degree < e + / — 2h. Factorizing 
by H, we get 

E/H ^ H ± /H (E/H)* 

F/H — G 0. 

We are in this situation precisely when the orthogonal extension H x /H with class 

qM*5(V)EH\X,A 2 (E/H)) 
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admits an isotropic lifting of some elementary transformation of (E/H)* of degree 
< e + f — 2h. By Criterion 12.81 (2). this is equivalent to the class S (H^/H) belonging 
to Sec5( e+ ^ 2/l )Gr(2, E/H) in FH\X, A 2 (E/H)). □ 

3. Parameter spaces of extensions 

Here we construct "universal extension spaces", following Lange [12] (see also [3l 
§4.2]), and investigate stability of the corresponding symplectic and orthogonal bun- 
dles. 

3.1. Construction of the families. For a positive integer e, let Ux{n, —e) s denote 
the moduli space of stable vector bundles of rank n and degree — e < over X. By 
Narasimhan-Ramanan [T6| Proposition 2.4], there exist a finite etale cover 

vr e : U e -> U x (n, -e) s 

and a bundle 8 e — > U e x X with the property that £ e \{E}xx — ^e{E) for all E G U e . 
When gcd(n, e) = 1, it is well known that ir e reduces to the identity map. 
By Riemann-Roch and semistability, for each E G Ux{n, — e) s , we have 

h\X, Sym 2 £) = (n + l)e + ^^(g - 1). 

Therefore, the sheaf i? 1 p*(Sym 2 (£ e )) is locally free of rank (n + l)(e + \n{g — 1)) on 
U e . We denote its projectivization by //: S e — > U e . We have a diagram 



E/e X X 




Ue X 

We write r: S e x X — > § e for the projection. By Lange [TU Corollary 4.5], there is 
an exact sequence of vector bundles 

0^(/iX Id x )*£ e ® r*C Se (l) -> W e -)• (/i x Id x )*4* -)• 

over § e x X, with the property that for 5 G S e with /i(<5) = £7, the restriction of V e to 
{5} x X is isomorphic to the extension of E* by E defined by 5 G ¥H 1 (X, Sjm 2 E). 
By Lemma 12.21 the space § e classifies all symplectic extensions of E* by E for all 
E E U(n, —e) s , up to homothety. 



16 



INSONG CHOE AND GEORGE H. HITCHING 



In the same way, we define a bundle A e — > U e whose fibre at E is FH 1 (X, A 2 E), 
a projective space of dimension (n — l)(e + \n(g — 1)) — 1. There is a sequence of 
vector bundles 

-> ( M x ld x )*S e <g) r*G Ae (l) -> V e -> (/i x Id x )*£ e * -> 

over A e x X, with the property that for 5 e A e with fi(5) = E, the restriction of W e 
to {5} x X is isomorphic to the extension of E* by E defined by 5 6 Pi7 1 (X, A 2 i?). 
Again by Lemma 12.21 the space A e classifies all the orthogonal extensions of E* by 
E for all E G U(n, —e) s , up to homothety. 

For each e > 0, the universal bundles W e — > S e x X and V e — > A e x X induce 
classifying maps 

<r e : § e —* MS x (2n) and a e : A e — > MO x (2n) 

respectively. The indeterminacy loci of these maps consist of precisely the points 
whose associated symplectic/orthogonal bundles are not semistable. 

3.2. Stability of extensions. The universal extension spaces S e and A e provide 
a natural way to study the Segre stratification on the moduli space of symplec- 
tic/orthogonal bundles, via the classifying maps a e and a e . In order to proceed in 
this direction, we must verify that a general bundle with extension class represented 
in § e (resp., A e ) is a stable symplectic (resp., stable orthogonal) bundle. 

The same question for vector bundles was formulated by Lange and solved in 
Brambila-Paz-Lange [2] and Russo-Teixidor i Bigas [19] . In both papers, elementary 
transformations were used to construct stable bundles with the prescribed Segre 
invariant. In this subsection, we prove the existence of a stable orthogonal/symplectic 
bundle in S e and A e for each e > 0. Elementary transformations are used, but in a 
somewhat different context. 

We begin by establishing the statement for e = 1 and e = 2. We will need the 
following bound on the classical Segre invariants of vector bundles, due to Hirschowitz 
[U Theoreme 4.4]: 

Lemma 3.1. Let E be a general stable vector bundle of rank n and degree —e. Let 
H C E be a subbundle of rank r and degree —h. Then 

r{n — r){g — 1) < nh — re < r(n — r)(g — 1) + n. 
Also we need the following result called the Hirschowitz lemma and its variant. 
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Lemma 3.2. 

(1) Let Hi and if 2 be general stable bundles, rk(ifj) = r, and deg(Hi) = di for 
i = 1,2. If rid 2 + r 2^i > r i r 2(g — 1) so £/ia£ <8> # 2 ) > — 1; 
h 1 (X,H 1 ®H 2 ) = 0. 

(2) Lei F be a general stable bundle of rank n. If degF > \n(g — 1) so that 
fi{F ®F)>g-l, then h 1 (X,F®F) = 0. 

Proof. Part (1) was proven by Hirschowitz jH 4.6]; see also Russo-Teixidor i Bigas 
[Ti?t Theorem 1.2]. The variant (2) is [2J Lemma Al]. □ 

Proposition 3.3. Let E e U(n, —e) s be general, e = 1,2. 

(1) For e = 1, every point in ¥H 1 (X, Sym 2 E) outside a sublocus Y with dimF < 
n = dimPF corresponds to a stable symplectic bundle. Hence a general point 
of Si represents a stable symplectic bundle. 

(2) For e = 1, every point in FH 1 (X, A 2 E) corresponds to a stable orthogonal 
bundle. For e = 2, every point of Pi/ 1 (X, A 2 E) outside a sublocus Z with 
dimZ < 2{n — 2) + 1 = dimGV(2, £") corresponds to a stable orthogonal 
bundle. Hence a general point of Ai or A 2 represents a stable orthogonal 
bundle. 

Proof. Consider a symplectic or orthogonal extension 0— > E — > V — > E* — > where 
E is a general stable bundle of rank n and degree — e G { — 1, —2}. Assume that V is 
not stable, so there is an isotropic subbundle F of V of rank r(< n) and degree > 0. 
The intersection of E and F contains a subbundle Fx of rank r\ (possibly zero), and 
the image of F in E* is a locally free subsheaf F2 of rank r 2 , yielding a diagram 

(3.1) ^ E *~ V E* *■ 

> Fi F F 2 0. 

We will bound the dimension of the locus of the extensions V admitting this kind of 
diagram. 

First assume r\ 7^ and so r 2 < n. Since E is general, by Lemma [3. II we have 

(3.2) de g j p 1 <-^(e + (n-r 1 )(^-l)) < 0. 

n 

In the same way (if r 2 7^ 0), 

(3.3) degF 2 <^(e-(n-r 2 )(#-l)), 
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which implies degF 2 < since r 2 < n. Therefore, deg-F < 0. 

Next, assume r\ = and r = r 2 = n. In this case, F is an elementary transfor- 
mation of E*. If degF > 0, then the torsion sheaf E/F has degree < e. In the 
symplectic case, we need only consider the case when e = 1. By Criterion 12.81 (1), 
if F lifts to V as a Lagrangian subbundle, then 5(V) E <p s (FE) in FH 1 (X, Sym 2 F). 
In the orthogonal case, e < 2. By Criterion 12.81 (2), if F lifts to V as an isotropic 
subbundle, then in fact e = 2 and 5{V) G Gr(2, E) in FH\X, A 2 E). 

Finally assume r± — and r = r 2 < n. From the inequality (13. 3p . deg F = deg F 2 < 
0. The possibility deg F = appears only for the following special cases: 

(i) e = 1; r = n — 1; g = 2 

(ii) e = 2; r = n — 1; g = 3 
(hi) e — 2; r — n — 1; g — 2 
(iv) e = 2; r = n — 2; g = 2. 

From now on, we show that in each of these cases, the dimension of the locus of 
extensions admitting a lifting of a subsheaf F of E* as an isotropic subbundle of V 
of degree zero and rank r < n is bounded by n = dimPF in Pif 1 (X, Sym 2 F) and 
2(n - 2) + 1 = dimGV(2, E) in ¥H\X, A 2 E) respectively 

Note that in each case (i)-(iv), F is a maximal subbundle of E*, which is easily seen 
from the inequalities in Lemma \3. II Hence the quotient Q := E*/F is torsion-free of 
degree e and we obtain the diagrams 

(3.4) E (F x )* Q -0 

E V + E* 

F — ^ F 

and 

(3.5) Q* G Q 

> Q* F 1 - E* ^ 

F — F. 
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Since {F ± ) ± = F, the bundle G = F /F inherits the nondegenerate bilinear form 
from V. Since F 1 - fl E = Q* is contained in the isotropic subbundle E C V, it is 
an isotropic subbundle of G. Thus the class of the extension G belongs to either 
H\X, Sym 2 Q*) or H^X, A 2 Q*) by Criterion O 

For a given general E, there are finitely many choices for F in cases (i), (ii), and 
(iv), while in case (iii), the subbundles of degree zero and rank n — 1 in E vary in a 
Quot scheme of dimension n — 1. Once F is chosen, the quotient Q = E*/F is fixed. 
After we choose a symplectic or orthogonal extension G of Q by Q*, the bundles F 1 - 
and V are determined from the above diagrams by 

(F x )* = (E®G) /Q* 

and 

V = [E © F ±s j /Q*. 

Therefore, for a fixed general E, the dimension of the locus of V appearing in the 
above class of diagram is bounded by that of the deformations of F and G. 

Let us consider the orthogonal case first. In cases (i)-(iii), the bundle Q has 
rank 1, so there is no nontrivial orthogonal extension of Q by Q*. Hence the only 
possibility for V is the direct sum E © E*, which is excluded. In case (iv), we have 
h}{X, A 2 Q*) = 3, and so 

dim{F : F C E*} + h\X, A 2 Q*) -1 = 2. 

Since < r = n — 2, we have 2 < 2(n — 2) + 1 as was claimed. 

For the symplectic case, we assumed e = 1, so (i) is the only case to be considered. 
In this case, there are finitely many choices of F, and h}(X, Sym 2 Q*) — 1 = 1 < n, 
as was claimed. 

This confirms that a general point of Sj, Ai or A 2 represents a stable orthogonal 
bundle. □ 

Theorem 3.4. For each e > 0, a general point of S e (resp., A e ) represents a stable 
symplectic (resp., orthogonal) bundle. 

Proof. We consider the orthogonal case first: For each value of e > 0, we will exhibit 
a stable orthogonal bundle represented in A e . The statement will then follow from 
the openness of the stable objects in families. 

Let E G Ux(n, — 1) be general. For any k > 2, choose a general antisym- 
metric principal part p of degree 2k, which defines an element of Sec fc Gr(2, E). 



20 



INSONG CHOE AND GEORGE H. HITCHING 



Since Gr(2, E) is nondegenerate and properly contained in Sec fe Gr(2, E), we may 
assume that [p] does not lie on the image of Gr(2,E). By Lemma \2.2\ the sheaf 
F := Ker (p: E* — > Prin (ET)) lifts to a rank n isotropic subbundle of the extension 
V with S(V) = [p\. Deforming p if necessary, we can assume that F is stable. By 
Proposition 13.31 (2), moreover, V is a stable orthogonal bundle. Since V fits into an 
orthogonal extension 

(3.6) 0^F^V-^F*^0, 

it is associated to a point in A2&-1. 

In the same way, consider a general bundle E G Ux{n, —2) and choose a general 
antisymmetric principal part of degree 2k + 2 for each k > 2, defining a point of 
Sec fc+1 Gr(2,£). By Proposition [331 (2), we may assume that the extension V asso- 
ciated to the point [p] is a stable orthogonal bundle, if we avoid a subvariety Z with 
dimZ < dimGr(2, E); and this is possible since Gr(2, E) is nondegenerate and prop- 
erly contained in Sec fc+1 Gr(2, E). As in the previous case, V fits into an orthogonal 
extension (13. 6 j) where this time deg(F) = — 2fc, so V is represented in A 2 fc. 

In the symplectic case, we argue similarly. For each k > 2, we can choose a general 
symmetric principal part q of degree k + 1 which defines a point of Sec fc+1 P£'. By 
the above argument, the extension V determined by q is a stable symplectic bundle 
which is represented in □ 

4. Description of the Segre strata for symplectic bundles 

In this section, we use the map o e : § e — -> .MS^^n) discussed in the previous 
section to prove Theorem ll.il Recall that for each positive even integer t, we denote 
by M.Sx(2ri]t) the sublocus of A4Sx(2n) consisting of symplectic bundles V with 
t(V) = t. Also, for V E AiSx(2n), we write M(V) for the space of Lagrangian 
subbundles of V of (maximal) degree — \t{V). 

Theorem 4.1. Consider a positive even integer t = 2e < n(g — 1) + 1. 

(1) A general point ofS e corresponds to a bundle V with t(V) = t; in particular, 
AiSx{2n\t) is nonempty. Furthermore, AiSx{2n;t) is irreducible. 

(2) If t < n{g — 1), then M(V) is a single point for a general V G AiSx{2n; t). 

(3) Ift G {n(g — 1), n(g — 1) + 1} and V is general in A4Sx{2n; t) then any pair 
of Lagrangian subbundles in M(V) intersect transversely in V in a general 
fiber. 
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Remark 4.2. Statements (2) and (3) can be viewed as a symplectic analogue of 
Lange-Newstead [HI Theorem 2.3 & Proposition 2.4]. 

Proof. Let E be a general bundle in U (n, — e) s for e > 1. By Theorem 13 .4[ the moduli 
map $ e \ E = FH\X,Sjm 2 E) — > A4S x (2n) is defined on a dense subset. We want 
to compute a bound on the dimension of the locus in H 1 (X,Sym 2 E) of extensions 
V which admit a Lagrangian subbundle F of degree > — e other than E. There are 
two possibilities: either F is an elementary transformation of E* lifting to V, or F 
fits into a diagram of the form (12. 6p . 

Step 1. We show first that the latter situation does not arise for a general V in 
H X {X, Sym 2 £) for t = 2e < n(g - 1) + 1. As before, we write deg(if) = -h. Recall 
from the diagram (I2.6P that G is an elementary transformation of (E/H)* whose 
quotient is a torsion sheaf of degree 

deg(E/H)* - degG = (e - h) - (deg F + h) < 2(e - h). 

In particular, 

(4.1) e-h>0. 

Now for each fixed H C E, Criterion 12.111 says that the locus of extensions in 
H l (X, Sym 2 E) admitting a diagram of the form ( 12 .6p for some F of degree > — e is 
bounded by 

dim (Sec 2ie - h) F(E/H)^) + 1 + dimKer (q* V) . 
The secant variety has dimension bounded by 

2(e - h){n - r) + 2(e - /i) - 1 = 2(e - /i)(n - r + 1) - 1. 

Also by Proposition 12.101 we have 

dimKer (qjq*) = h\X, Sym 2 E) - h 1 (X, Sym 2 (E/H)) 

< h\X, Sym 2 E) - (n - r + l)(e - h) - -(n - r)(n - r + l)(g - 1). 

Finally we take account of the deformations of H by computing the dimension of the 
appropriate Quot scheme of E. Since E is general, by Lemma [3. II we have 

nh — re > r(n — r)(g — 1). 

We may furthermore assume that H and E/H are general, and so 

h l {X,Yiom(H,E/H)) = 
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by Lemma I3T21 (1). Therefore [H] is a smooth point of the Quot scheme (cf. Le Potier 
[T5| p. 125]), and the dimension of the Quot scheme is given by 

h°(X, Hom(F, E/H)) =nh~re- r(n - r)(g - 1). 

Adding up these three terms, we see that the dimension of the locus of extensions in 
H X (X, Sym 2 E) admitting a diagram of the form (12 .6p for some F of degree > — e is 
bounded by 

dim (Sec 2(e_/,) P(£/#)) + 1 + dimKer(g* V) + h°(X, Hom(F, E/H)) 

< (e - h)(n - r + 1) - ~(n - r)(n + r + l)(g - 1) +nh - re + h x (X, Sym 2 £) 

= e(n - r) - (e - h)(r - 1) - ~(n - r)(n + r + l)(g - 1) + ^(X, Sym 2 E). 

By the assumption e < ^(^(fl 1 — 1) + 1) anc ^ ^ ne inequality (14. ip . this is smaller 
than the dimension of the whole extension space h r (X, Sym 2 E). This shows that for 
t = 2e < n(g — 1) + 1, a general G Sym 2 ^) does not admit a Lagrangian 

subbundle F of degree > — e which fits into a diagram of the form (12.61) . 

Step 2. Next we consider the case r = 0, so F is an elementary transformation of 
E* lifting to V isotropically. Let degF = — / > — e. By Criterion 12. 8[ the dimension 
of the locus of extensions in H l (X, Sym 2 ^) admitting such an F is bounded by 

dim (Sec e+/ P£) + 1 < (e + f)(n + l). 

If / < \n(g — 1), this bound is smaller than 

h\X, Sym 2 E) = (n+ l)e + ^n(n + l){g- 1). 

Step 3. Combining the dimension counts for two possibilities in the above, we 
conclude: 

(i) if t = 2e < n(g — 1), a general V E Sym 2 ^) does not have a Lagrangian 
subbundle of degree > — e other than E itself. This shows (2). 

(ii) if t = 2e E {n(g — 1), n(g — 1) + 1}, a general V E H X (X, $ym 2 E) does not have 
a Lagrangian subbundle of degree > — e. Also, any Lagrangian subbundle of degree 
— e different from E intersects E transversely at a general fiber. This shows (3). 

Hence for a general V E H X (X, Sym 2 E) with t = 2e < n(g — 1) + 1, we have 
t(V) = t. It follows that the rational map cr e : § e --• * A4Sx{2n) sends a general 
point of S> e to A4Sx(2n;t), so J\4Sx(2n',t) is non-empty for each t < n(g — 1) + 1. 
Since S e is irreducible, so is its image. To see that .MSx(2n;t) is irreducible, it 
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suffices to show that every point of M.Sx(2n; t) is in the closure of the image of S e . 
Any V in A4Sx{2n;t) admits a Lagrangian subbundle E of degree — e which might 
be unstable. But every such E is contained in an irreducible family of bundles whose 
general member is a stable bundle in Ux{n, — e). This shows (1). □ 



To finish the proof of Theorem II .![ we need the following description of the tangent 
space of M(V). 

Lemma 4.3. For a symplectic bundle V £ AiSx{2n) and a point [rj : E C V] of 
M{V), the Zariski tangent space of M(V) at i] is identified with H°(X, Sjm 2 E*). 

Proof. It is well known that the Zariski tangent space to the Quot scheme of a vector 
bundle V at a point [E C V] is given by H°(X, Hom(£', V/ E)). Intuitively, this can be 
explained as follows: A rank n subbundle of V is equivalent to a global section of the 
Grassmannian bundle Gr(n, V) over X. A tangent vector to the Quot scheme of V at 
[E C V] corresponds, at each fiber x £ X, to a tangent vector to the Grassmannian 
Gr(n, V x ) at [E x C V x }. Therefore, the tangent vector is a global section of the 
bundle Hom(£', V/E). When E is a Lagrangian subbundle of a symplectic bundle V, 
we have V/E = E* and so Hom(£', V/E) = E*Z$>E* . In this similar argument 

shows that a tangent vector to M(V) at [E C V] corresponds to a global section of 
Sym 2 £*, since for each x £ X the tangent space of the Lagrangian Grassmannian at 
[E x C V x ) is identified with Sym 2 E*\ x . □ 

Proof of Theorem li.il 

We begin with part (3) of the statement. The first part of (3) is Theorem 14.11 (2). 
For the latter part, we invoke Lemma I4T31 dimM(V) = h°(X, §ym 2 E*) for a smooth 



point [E C V] of M{V). By Lemma [3.21 (2), for a general E £ Ux(n, — e), we have 

fo if2e = n(o-l), 
dimH°(X,Sym 2 £*) = { W ' 

if 2e = n(g - 1) + 1. 

(1) A straightforward computation shows that when n(g — 1) < 2e < n{g — 1) + 1, 

dimM(V) = dim§ e - dim MS x (2n). 

Since dimM(V) > dima~ 1 (V A ), this equality implies that a e is dominant. This shows 
that for a general V £ A^S , x(2ra), we have 

n(g-l)<t(V)<n(g-l) + l. 
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By semicontinuity, t(V) < n(g — 1) + 1 for any symplectic bundle V. 

(2) If * = 2e > n(g - 1), then dim MS x (2n; t) = dimMS x (2n) by (1). Assume 
t = 2e < n(g — 1). By Theorem 14.11 (2), the map a e : § e --• * A4Sx{2n;t) is generi- 
cally finite (of degree deg7r e ), and so 

dimMS x (2n;t) = dimS e = -n(3n + l)(g - 1) + (n + l)e. 

For the last part of (2), we can use Criterion [23] to show that the Segre stratification 
matches the stratification given by the higher secant varieties: For a general V G 
MS x (2n; 2e), let E G M(V) so that E is general in U x (n, -e) s . Let A; be the 
smallest integer satisfying Sec fc PF = FH 1 (X,Sym 2 E). Then by Criterion 12. S\ there 
is some elementary transformation F of E* with degF/F = /c lifting to V as a 
Lagrangian subbundle. By deforming V and E inside S e , we may assume that F is 
general in Ux{n, e — k) s . Now consider the symplectic extension 

0^F^V^F*^0. 

In the same way, S(V) G FH^X^ju^F) belongs to Sec fc PF, since the elementary 
transformation E — >■ F* lifts to V^. Note that 

dimSec fe PF < jfc(n+l)-l < (n+l)(A;-e)+-n(n+l)(^-l)-l = dimFH\X, Sym 2 F), 

since we assumed 2e < — 1). Therefore, Sec fc PF is properly contained in 
FH\X, Sym 2 F), and certainly it is inside the closure of Sec fc+1 PF \ Sec fc PF. Thus, 
again by Criterion I2.8[ the class S(V) belongs to the closure of a family of bundles 
admitting liftings of elementary transformations of degree (k—e) — (k+l) = — (e+1). 
In particular, V belongs to the closure of A4Sx{2n;2e + 2) in A4Sx(2n). By the 
irreducibility of AiSx(2n; 2e), the same holds for arbitrary V G A4Sx(2n; 2e). □ 

Remark 4.4. Suppose t = 2e < n(g — 1). If gcd(n, e) = 1, then U e = U x (n, — e) s 
and AiSx{2n; t) is birational to the fibration S e over Ux{n, —e) s whose fiber at E is 
FH\X, Sym 2 F). 

Finally we give a geometric interpretation of the cardinality of M(V), when it 
is finite. Assume t = 2e = n(g — 1). Let V be general in A4Sx(2n;t) and 
choose E G M(V) with degF = — e = —^n(g — 1). Consider the subvariety 
FE C Pi/^A^Sym 2 ^). When g > 4, we have /j(E) < -1, and by Lemma 1231 
the map <p s : FE — > Pff 1 (A, Sym 2 F) is an embedding. Furthermore, it was proven 
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in [31 Proposition 3.2] that the secant variety FE is not defective in FH 1 (X, Sym 2 E) 
in the sense that 

dimSec fc P£ = min{(n + l)k - 1, dimFH\X, Sym 2 £)}. 

In particular, for k — n(g — 1) we have 

(4.2) dimSec fe P£ = (n + l)k-l = dimPF^X, Sym 2 £). 

By a k-secant space of FE, we mean a linear subspace in FH 1 (X, Sym 2 ^) spanned 
by some k points of FE. By (14.21) . there are finite number of n(g — l)-secant spaces 
of FE which pass through a general point of FH 1 (X, Sjm 2 E). The following result 
generalizes Lange-Narasimhan [131 Proposition 2.4] for rank 2 bundles. 

Theorem 4.5. Assume g > 4 and n(g — 1) is even. Let V G A4Sx{2n) be a general 
symplectic bundle with t(V) = n(g — 1), and let E G M(V). There is a one-to-one 
correspondence between M(V)\{E} and n(g — l)- secant spaces ofFE passing through 
the point [E C V] G FH^X, Sym 2 £). 

Proof. By Criterion I2.8[ a general class S(V) lies on a n(g — l)-secant space of 
FE if and only if there is an associated elementary transformation F of E* with 
deg(E* /F) = n(g — l), lifting to a Lagrangian subbundle of V. Since t(V) = n(g — l), 
the Lagrangian subbundle F is an element of M(V). Conversely, by Theorem 14.11 (3) 
every element of M(V) other than E appears in this way, since V is general. □ 

Remark 4.6. It is an interesting problem to compute the cardinality of M(V) ex- 
plicitly. Let N n>g denote the cardinality of M(V) for a general symplectic bundle V 
of rank 2n over a curve of genus g so that n(g — 1) is even. It is well known that 
N lt9 = 2 9 . The same problem for maximal subbundles of vector bundles was solved 
by Holla [9j (see also Lange-Newstead [13]). As far as we are aware, the number N n>g 
is not known in general. 

5. Description of Segre strata for orthogonal bundles 

In this section, we investigate the geometry of the Segre stratification on the mod- 
uli space of orthogonal bundles AiOx(2n). In contrast to AiSx(2n), this has two 
connected components A / 10x(2n) ± , corresponding to bundles of trivial and nontriv- 
ial second Stiefel-Whitney class (see Serman [20] for more details). We begin by 
determining the component to which each stratum AiOx{2n;t) belongs. 
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5.1. Topological classification. 

Theorem 5.1. (1) Let E\ and E 2 be isotropic rank n subbundles of an orthogonal 
bundle V of rank In. Then deg-Ei and degE 2 have the same parity. 
(2) A semistable orthogonal bundle V belongs to AiOx(2n) + (resp., A40x{2n)~ ) if 
and only if its isotropic rank n subbundles have even degree (resp., odd degree). 

Proof. By Theorem 13 .4} the moduli maps a e : A e — - > AiOx{2n) are defined on dense 
subsets for each e > 0. Since each family A e is connected, the image of each A e 
is entirely contained in either M.Ox{^n) + or .MOx(2n)~. As was already noted in 
the proof of Theorem 13.41 whenever e\ and e% have the same parity, the images of 
A ei and A e2 have nonempty intersection. Hence they lie on the same component of 
M.0 x(2n) . This is essentially due to the fact that any antisymmetric principal part 
has an even degree (see Lemma 12.61) . 

For each k > 0, the image of A 2 k is contained in M.Ox{^n) + since the trivial 
bundle has trivial second Stiefel- Whitney class. Thus the image of A2fe_i is contained 
in AiOx{2n)~ . This shows the above statements (1) and (2) for the case when 
V is a stable orthogonal bundle and the isotropic subbundles are stable bundles. 
By deforming an arbitrary bundle to a stable one, we see that (1) and (2) hold in 
general. □ 

5.2. An extra stratum. The local deformations of an orthogonal bundle V are 
given by H X (X, A 2 V). Since V is self-dual, A 2 V C End(V r ). Hence both components 
A / fOx(2n) ± have dimension 

h\X, AV) =n(2n- l)(g - 1), 
since we may assume that V is simple. On the other hand, 

dim A e = dim U x (n,-e) + dim FH 1 (X, A 2 E) 
= (n-l)e+in(3n-l)(0-l) 

for a general E 6 Ux{n, —e). Comparing dimensions, we see that in order for an 
A e to cover either component of AiOxi^n), we need e > \\n{g — 1)]. However, a 
connected set A e can cover at most one component. Hence we are led to consider 
at least two top strata, corresponding to distinct values of e > \ \n{g — 1)] . This is 
one important difference between the stratifications in the symplectic and orthogonal 
cases. 
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5.3. Geometry of the strata. The arguments in this subsection are essentially 
the same as those appearing in §4, except for the complication coming from the 
topological property described in Theorem 15.11 

Theorem 5.2. Consider a positive even integer t = 2e < n(g — 1) + 3. 

(1) A general point of A e corresponds to a bundle V with t(V) = t; in particular, 
A40x{2n]t) is nonempty and irreducible. 

(2) If t < n(g — 1) then M(V) is a single point for general V G MOx(2n;t). 

Proof. Let E be a general bundle in U(n, —e) s for e > 1. As before, we will bound 
the dimension of the locus in H l (X,A 2 E) of extensions V which admit a rank n 
isotropic subbundle F of degree > — e other than E. Again, either F is an elementary 
transformation of E* lifting to V, or F fits into a diagram of the form (12.61) . 

Step 1. We first show that the latter situation does not arise for a general V in 
A 2 E) for t < n(g — 1) — 1 (not t < n(g — 1) + 1 as in the symplectic case; see 
Remark 15. 3p . For each fixed H C E, by Criterion 12.111 (2), the locus of extensions 
fitting into a diagram of the form (12.61) for some F of degree > — e is bounded by 



dim (Sec e - h Gr(2, E/H)) + 1 + dim Ker (q* *q*) . 



As in the symplectic case, we obtain e — h > 0. Since 



dim Gr(2, E/H) = 2(n - r - 2) + 1 



2(n - r) - 3, 



the secant variety has dimension bounded by 



(e - h)(2(n - r) - 3) + (e - h) - 1 = 2(e 



h)(n — r 



1)-1. 



By Proposition 12.101 we have 



dim Ker (q, Y) = h\X, A 2 E) 
< h x {X, A 2 E) 



h 1 (X,A 2 (E/H)) 



(n — r — l)(e — h) 



(n — r)(n — r — l)(g — 1). 



As before, the subbundle H of E varies in a Quot scheme of dimension 



h°(X, Hom(if, = nh - re - r{n - r)(g - 1). 
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Thus the dimension of the locus of extensions in H 1 (X,A 2 E) admitting a diagram 
of the form (12.60 for some F of degree > — e is bounded by 

dim (Sec e ^Gr(2, E/H)) + 1 + dim Ker(g, Y) + h°(X, Hom(#, E/H)) 

< (e - h){n - r - 1) - -(n - r)(n - r - 1)(# - 1) + nh - re+ h l {X, A 2 E) 

= e(n — r) — (e - h){r + 1) - -(n - r)(n - r - l)(g - 1) + h\X, A 2 E). 

Since e — h > and we have assumed e < |(ti(<7 — 1) — 1), this is smaller than 
h x (X, A 2 E). Therefore, for t = 2e < n(g - 1) - 1, a general V G H^X, A 2 E) does 
not admit a rank n isotropic subbundle F of degree > — e which fits into a diagram 
of the form fl23|) . 

.Ste;? ^. Next we consider the case r = 0, so F is an elementary transformation 
of .E* lifting to V isotropically. Write deg F = —f > —e. By Criterion 12.21 (3) and 
Lemma 12. 6\ we have e + / = mod 2. By Criterion 12.81 (2), the dimension of the 
locus of extensions in A 2 ^) admitting such an F is bounded by 

dim(Sec5 (e+/) Gr(2,£)) +1 < -(e + f)(2(n - 2) + 2) = (e + f)(n - 1). 

If / < — 1), this bound is smaller than 

h\X, A 2 E) = (n - l)e + Ki(n - \){g - 1) 

so a general extension in H 1 (X, A 2 E) does not admit any such lifting. 

Step 3 of the proof of Theorem 14. II carries over, mutatis mutandis, to the orthogonal 
case, to prove statement (2), and (1) for t < n(g — 1) — 1. To prove (1) for n(g — 1) < 
t < n(g — 1) + 3, we just repeat the arguments in Steps 1 and 2 under the assumption 
— / > — e + 2, to show that for general E e Ux{n, — e), a general V G H l (X, A 2 E) 
does not admit a rank n isotropic subbundle F of degree > — e + 2. □ 

Remark 5.3. Note that we do not prove an analogue of Theorem 14.11 (3) for the 
orthogonal case. A dimension count analogous to that in the proof of Theorem 14.11 
(3) does not exclude the possibility that two maximal isotropic rank n subbundles of 
a general bundle in A40x(2n) intersect in a line bundle, or in a rank 2 subbundle if 
g = 2. It is unclear to us at this stage whether or not this in fact happens. 

Next, we compute the dimension of M(V) for a general V G A / 10x(2n) ± . We will 
use the following analogue of Lemma I4.3[ which is proven in the same way as in the 
symplectic case: 
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Lemma 5.4. For an orthogonal bundle V G M.Ox{2n) and a point [i] : E C V] of 
M(V), the Zariski tangent space of M(V) at 77 is identified with H°(X, A 2 E*). □ 

Combining with Lemma [3.21 (2). we obtain: 

Corollary 5.5. Let t = 2e = n(g — 1) + e with e G {0, 1, 2, 3}. Then for a general 
V G MO x (2n;t), we have 

dimM(\/) = £(n ~ 1) . 

□ 

Proof of Theorem \1.3[ 



(1) Consider the rational map a e : A e — > AiO x (2n). For each t = 2e = n(g — 1) + e 
where e G {0, 1, 2, 3}, a direct computation shows that 

dimM(y) = dimA e - dim MO x {2n). 

Since dimM(V) > dima~ l (V) for any V G AiOx{2n; 2e), this equality implies that 
a e is dominant to a component of AiO x(2n) . This shows that for general V in some 
one of the components A^Ox(2n) ± , we have 

n(g-l)<t(V)<n(g-l) + 3. 

By semicontinuity, t(V) < — 1) + 3 for any orthogonal bundle V. 

(2) The nonemptiness and irreducibility were proven in Theorem 15.21 (1). The re- 



maining part can be proven in the same way as Theorem 11.11 (2). 



(3) This was proven in Theorem 15.21 (2). 



□ 



5.4. Configuration of the dense strata. For each 2e = n(g— l)+e with < e < 3, 
the locations of the images of the a e depend on the congruence class of n(g— 1) modulo 
4: the trivial bundle of rank 2n is contained in A40x{2n) + , hence for each k, we 
have MO x (2n;Ak) C MO x (2n)+ and MO x (2n;Ak + 2) C «MO x (2n)-. We may 
summarize the situation for the dense strata as follows: 



n(g — 1) = mod 4 : 



t 


Component 


dimM(V) 


n(g - 1) 


MO x (2n) + 





n(g-l) + 2 


MO x (2n)- 


|(n-l) 
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n(g — 1) = 1 mod 4 : 



n(g — 1) = 2 mod 4 : 



t 


Component 


dimM(V) 


n( a — 


I 


+ 1 


MOx{2n)- 


h(n-l) 


n(g - 


r 


+ 3 


MO x {2n) + 


n — 1 




t 




Component 


dimM(V) 


n(g 




1) 


MO x (2n)- 





n(g - 


r 


+ 2 


MO x {2n) + 






t 




Component 


dimM(V) 


n(g - 


i 


+ 1 


MO x {2n) + 




n(g - 


r 


+ 3 


MO x (2n)- 


?2 — 1 



— 1) = 3 mod 4 : 



Remark 5.6. In J3J §4], symplectic bundles W of rank four were studied which 
satisfy S2(W) < t(W); that is, whose maximal vector subbundles of half rank are all 
nonisotropic. Orthogonal bundles V belonging to the top stratum AiOx{2n; n(g — 
1) + 2) provide a different example of this phenomenon. Due to the Hirschowitz 
bound all such V have a vector subbundle of degree at least — \\n(g — 1)] , but 
no isotropic rank n subbundle of this degree or greater. 

When n(g — 1) is even, such an orthogonal V is nongeneral as a vector bundle 
(compare with Lange-Newstead [T4"l §2]): Since any maximal rank n subbundle F C 
V is nonisotropic, we must have h°(X, Sym 2 F*) > 0. But in this case /i(Sym 2 F*) < 
g — 1. By Lemma [3.21 (2), this is a condition of positive codimension on F, so none 
of the maximal vector subbundles of V are general. 
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